Robust human tracking based on multi-cue integration and mean-shift

Multi-cue integration has been researched extensively for robust visual tracking. Researchers aim to use multiple cues under the probabilistic methods, such as Particle Filtering and Condensation. On the other hand, Color-based Mean-Shift has been addressed as an effective and fast algorithm for tracking color blobs. However, this deterministic searching method suffers from objects with low saturation color, color clutter in backgrounds and complete occlusion for several frames. This paper integrates multiple cues into Mean-Shift algorithm to extend its application areas of the fast and robust deterministic searching method. A direct multiple cues integration method with an occlusion handler is proposed to solve the common problems in color-based deterministic methods. Moreover, motivated by the idea of tuning weight of each cue in an adaptive way to overcome the rigidity of the direct integration method, an adaptive multi-cue integration based Mean-Shift framework is proposed. A novel quality function is introduced to evaluate the reliability of each cue. By using the adaptive integration method, the problem of changing appearance caused by object rotation can be solved. Extensive experiments show that this method can adapt the weight of individual cue efficiently. When the tracked color blob is invisible for human bodies’ rotation, the color cue is compensated by motion cue. When the color blob becomes visible again, the color cue will become dominating as well. Furthermore, the direct-cue-integration method with an occlusion handler is combined with the adaptive integration method to extend the application areas of the adaptive method to full occlusion cases. 2008 Published by Elsevier B.V.